International  Journal  of  Applied  Engineering 
Research  and  Development  (IJAERD) 
ISSN(P):  2250-1584;  ISSN(E):  2278-9383 
Vol.  4,  Issue  2,  Apr  2014, 113-122 
©  TJPRC  Pvt.  Ltd. 

BLUE  EYES  SENSING  INTELLIGENCES  TECHNOLOGY  USING  EMOTION  SENSOR 

PRIYA  R.  BAGHE 

UG  Students,  Information  Technology,  VIT,  Wadala,  Mumbai,  Maharashtra,  India 

ABSTRACT 

The  BLUE  EYES  technology  is  like  a  non-obtrusive  sensing  method  for  creating  computational  machines  that 
have  perceptual  and  sensory  ability  like  those  of  human  beings.  This  commonly  uses  most  modern  video  cameras  and 
microphones  to  identify  the  user's  actions  through  the  use  of  imparted  sensory  abilities.  It  has  the  ability  special  techniques 
like  facial  recognition,  speech  recognition,  etc.  It  can  even  understand  your  emotions  at  the  touch  of  the  mouse.  It  verifies 
your  identity,  feels  your  presents,  and  starts  interacting  with  you. 

The  user  can  concentrate  on  observation  and  manual  operations,  and  still  control  the  machinery  by  voice  input 
commands.  A  Blue  Eyes-enabled  television  could  become  active  when  the  user  makes  eye  contact,  at  which  point  the  user 
could  then  tell  the  television  to  "turn  on".  Human  error  is  still  one  of  the  most  frequent  causes  of  all  artificial  disasters. 
Today  human  contribution  to  the  overall  performance  of  the  system  is  left  unsupervised.  This  paper  is  about  the  important 
modules,  techniques,  methodizes  and  relevance  updating  technologies  in  the  "blue  eye". 
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INTRODUCTION 

The  "BLUE  EYES"  technology  aims  at  creating  computational  machines  that  have  perceptual  and  sensory  ability 
like  those  of  human  beings.  How  can  we  make  computers  "see"  and  "feel"?  Blue  Eyes  uses  sensing  technology  to  identify 
a  user's  actions  and  to  extract  key  information.  This  information  is  then  analyzed  to  determine  the  user's  physical, 
emotional,  or  informational  state,  which  in  turn  can  be  used  to  help  make  the  user  more  productive  by  performing  expected 
actions  or  by  providing  expected  information.  Human  cognition  depends  primarily  on  the  ability  to  perceive,  interpret, 
and  integrate  audio-visuals  and  sensoring  information.  Adding  extraordinary  perceptual  abilities  to  computers  would 
enable  computers  to  work  together  with  human  beings  as  intimate  partners.  Researchers  are  attempting  to  add  more 
capabilities  to  computers  that  will  allow  them  to  interact  like  humans,  recognize  human  presents,  talk,  listen,  or  even  guess 
their  feelings.  Blue  eyes  system  monitors  the  status  of  the  operator's  visual  attention  through  measurement  of  saccadic 
activity.  The  system  checks  parameters  like  heart  beat  rate  and  Blood  oxygenation  against  abnormal  and  triggers  user 
defined  alarms. 

It  can  even  understand  your  emotions  at  the  touch  of  the  mouse.  It  verifies  your  identity,  feels  your  presents, 
and  starts  interacting  with  you  .You  asks  the  computer  to  dial  to  your  friend  at  his  office.  It  realizes  the  urgency  of  the 
situation  through  the  mouse,  dials  your  friend  at  his  office,  and  establishes  a  connection. 
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Figure  1:  Blue  Eyes 


WHAT  IS  BLUE  EYES  TECHNOLOGY? 

Aims  at  creating  computational  machines  that  have  perceptual  and  sensory  ability.  Use  camera  and  microphone  to 
identify  user  actions  and  emotions  Blue  Eyes  -  Bluetooth  technology  and  the  movements  of  the  eyes.  Bluetooth  provides 
reliable  wireless  communication.  As  the  eye  movements  enable  us  to  obtain  a  lot  of  Interesting  and  important  information. 
This  required  designing  a  Personal  Area  Network  linking  all  the  operators  and  the  Supervising  system. 

For  example,  a  Blue  Eyes-enabled  television  could  become  active  when  the  user  makes  eye  contact,  at  point  the 
user  could  then  tell  the  television  to  "turn  on". 

WHY  BLUEEYES? 

Human  error  is  still  one  of  the  most  frequent  causes  of  all  artificial  disasters.  Today  human  contribution  to  the 
overall  performance  of  the  system  is  left  unsupervised.  Since  the  system  is  made  to  perform  automatically,  an  operator 
becomes  a  passive  observer  of  the  supervised  system,  which  causes  drop  to  awareness.  It  therefore  is  crucial  to  assure  that 
the  operator's  conscious  brain  is  involved  in  an  active  system  which  will  supervise  over  the  whole  work  time  period. 
It  is  possible  to  measure  indirectly  the  level  of  the  operator's  conscious  brain  involvement  using  eye  movement  analysis. 
In  large  control  rooms,  wiring  the  operator  to  the  central  system  is  a  serious  limitation  of  his  mobility  and  disables  his 
operation.  The  wireless  link  between  the  sensors  worn  by  the  operator  and  the  supervising  system  offers  new  way  to 
system  overall  reliability  and  safety. 

•  To  built  a  machine  that  can  understand  your  emotions 

•  A  pc  that  can  listen,  talk  or  scream 

•  Verify  your  identity,  feels  your  presence  and  interact  with  you. 
The  Overall  System  Diagram  is  as  Follows 
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Figure  2:  The  Overall  System  Diagram  is  as  Follows 
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TECHNOLOGIES  USED 

•  Emotion  Mouse 

•  Manual  And  Gaze  Input  Cascaded  (MAGIC) 

•  Artificial  Intelligent  Speech  Recognition 

•  Simple  User  Interest  Tracker  (SUITOR) 

•  The  eye  movement  Sensor 
Types  of  Emotion  Sensors 

•  For  Hand 

•  Emotion  Mouse 

•  Sentic  Mouse 

•  For  Eyes 

•  Expression  Glasses 

•  Magic  Pointing 

•  Eye  Tracking 

•  For  Voice 

•  Artificial  Intelligence  Speech  Recognition 

Emotion  Mouse 

Rosalind  Picard  (1997)  describes  why  emotions  are  important  to  the  computing  community.  There  are  two  aspects 
of  affective  computing:  giving  the  computer  the  ability  to  detect  emotions  and  giving  the  computer  the  ability  to  express 
emotions.  Not  only  are  emotions  crucial  for  rational  decision  making  as  Picard  describes,  but  emotion  detection  is  an 
important  step  to  an  adaptive  computer  system.  An  adaptive,  smart  computer  system  has  been  driving  our  efforts  to  detect  a 
person's  emotional  state.  An  important  element  of  incorporating  emotion  into  computing  is  for  productivity  for  a  computer 
user.  A  study  (Dryer  &  Horowitz,  1997)  has  shown  that  people  with  personalities  that  are  similar  or  complement  each  other 
collaborate  well. 


Emotion  mouse 

Emotional  mouse  developed  at  IBM  Research  Lab 

implemented  on  a  real  mouse. 

Figure  3:  Emotion  Mouse 
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•  Simplest  way 

•  People  spend  approximately 

•  1/3  of  their  total  computer 

•  Time  touching  input  device 

•  Physiological  data  is 

•  Obtained  and  emotional 

•  State  is  determined 

•  A  user  model  will  be 

•  Built  that  reflects  the 

•  Personality  of  user. 

Samples  Obtained  from  Emotional  Mouse 
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Figure  4:  Samples  Obtained  from  Emotional  Mouse 

Expression  Glasses 

A  wearable  device  which  allows  any  viewer  to  visualize  the  confusion  and  interest  levels  of  the  wearer. 
Other  recent  developments  in  related  technology  are  the  attempt  to  learn  the  needs  of  the  user  just  by  following  the 
interaction  between  the  user  and  the  computer  in  order  to  know  what  he/she  is  interested  in  at  any  given  moment. 


Figure  5:  Expression  Glasses 


Impact  Factor  (JCC):  1.6842 


Index  Copernicus  Value  (ICV):  3.0 


Blue  Eyes  Sensing  Intelligences  Technology  Using  Emotion  Sensor 


117 


For  example,  by  remembering  the  type  of  websites  that  the  user  links  to  according  to  the  mood  and  time  of  the 
day,  the  computer  could  search  on  related  sites  and  suggest  the  results  the  user. 

MANUAL  AND  GAZE  INPUT  CASCADED  (MAGIC) 

Reduce  the  cursor  movement  needed  for  target  selection  Click  on  the  target  with  a  regular  manual  input  device 
•     Two  magic  pointing  techniques 

•  Liberal 

•  Conservative 


Eye  ball 
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Figure  6:  Manual  and  Gaze  Input  Cascaded  (MAGIC) 

The  liberal  MAGIC  pointing  technique:  cursor  is  placed  in  the  vicinity  of  a  target  that  the  user  fixates  on.  Actuate 
input  device,  observe  the  cursor  position  and  decide  in  which  direction  to  steer  the  cursor.  The  cost  to  this  method  is  the 
increased  manual  movement  amplitude.  The  conservative  MAGIC  pointing  technique  with  "intelligent  offset"  To  initiate  a 
pointing  trial,  there  are  two  strategies  available  to  the  user.  One  is  to  follow  "virtual  inertia:"  move  from  the  cursor's 
current  position  towards  the  new  target  the  user  is  looking  at.  This  is  likely  the  strategy  the  user  will  employ,  due  to  the 
way  the  user  interacts  with  today's  interface.  The  alternative  strategy,  which  may  be  more  advantageous  but  takes  time  to 
learn,  is  to  ignore  the  previous  cursor  position  and  make  a  motion  which  is  most  convenient  and  least  effortful  to  the  user 
for  a  given  input  device. 
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Figure  7:  Manual  and  Gaze  Input  Cascaded 
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APPROACHES 
Liberal  Approach 

To  warp  the  cursor  to  every  new  object  user  looks  at 

Conservative  Approach 

Does  not  warp  the  cursor  target  until 

ARTIFICIAL  INTELLIGENT  SPEECH  RECOGNITION 

Artificial  intelligence  (AI)  involves  two  basic  ideas.  First,  it  involves  studying  the  thought  processes  of  human 
beings.  Second,  it  deals  with  representing  those  processes  via  machines  (like  computers,  robots,  etc).  AI  is  behavior  of  a 
machine,  which,  if  performed  by  a  human  being,  would  be  called  intelligent.  It  makes  machines  smarter  and  more  useful, 
and  is  less  expensive  than  natural  intelligence.  Natural  language  processing  (NLP)  refers  to  artificial  intelligence  methods 
of  communicating  with  a  computer  in  a  natural  language  like  English.  The  main  objective  of  a  NLP  program  is  to 
understand  input  and  initiate  action.  The  input  words  are  scanned  and  matched  against  internally  stored  known  words. 
Identification  of  a  key  word  causes  in  some  action  to  be  taken.  In  this  way,  one  can  communicate  with  the  computer  one's 
language.  No  special  commands  or  computer  language  are  required.  There  is  no  need  to  enter  programs  in  a  special 
language  for  creating  software. 

Application  of  Artificial  Intelligent  Speech  Recognition 

•  To  control  weapons  by  voice  commands 

•  Pilot  give  commands  to  computers  by  speaking  into  microphones 

•  Can  be  connected  to  word  processors  and  instead  of  writing,  simply  dictate  to  them 

SIMPLE  USER  INTEREST  TRACKER  (SUITOR) 

Help  by  fetching  more  information  at  desktop  Notice  where  the  user's  eyes  focus  on  the  screen  fills  a  scrolling 
ticker  on  a  computer  screen  with  information  related  to  user's  task  ex.  If  reading  headline,  pops  up  the  story  in  the  browser 
window 

The  Eye  Movement  Sensor  Designing 

A  personal  area  network  for  linking  all  the  operators  and  the  supervising  system 
Two  major  units 

•  DAU  (Data  Acquisition  Unit) 

•  CSU  (Central  System  Unit) 
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Bluetooth  technology  provides 
means  for  creating  a  Personal  I 
Area  Network  linking  the 
operators  and  the  central 
system. 

Figure  8:  The  Eye  Movement  Sensor  Designing 
DAU  (DATA  ACQUISITION  UNIT) 

The  DAU  consists  of  the  following  components 
ATMEL  8952  microcontroller 

BLUE  TOOTH  MODULE  -  supports  synchronous  voice 
Data  transmission 
Central  system  sound  feedback 
Microcontroller  (1 15200  bps) 
ALPHAUNUMERIC  LCD  display 
LED  indicators 
ID  CARD  interface 

Data  Acquisition  Unit  is  a  mobile  part  of  the  Blue  eyes  system.  Its  main  task  is  to  fetch  the  hysiological  data  from 
the  sensor  and  to  send  it  to  the  central  system  to  be  processed.  To  accomplish  the  task  the  device  must  manage  wireless 
Bluetooth  connection  as  (connection  establishment,  authentication  and  termination).  Personal  ID  cards  and  PIN  codes 
provide  operator's  authorization.  Communication  with  the  operator  is  carried  on  using  a  simple  5-key  keyboard, 
a  small  LCD  display  and  a  beeper.  When  an  exceptional  situation  is  detected  the  device  uses  them  to  notify  the  operator. 
Voice  data  is  transferred  using  a  small  headset,  interfaced  to  the  DAU  with  standard  mini-jack  plugs. 

CSU  (Central  System  Unit) 

Central  System  Unit  hardware  is  the  second  peer  of  the  wireless  connection.  The  box  contains  a  Bluetooth 
module  (based  on  ROK101008)  and  a  PCM  codec  for  voice  data  transmission.  The  module  is  interfaced  to  a  PC  using  a 
parallel,  serial  and  USB  cable.  The  audio  data  is  accessible  through  standard  mini-jack  sockets  to  program  operator's 
personal  ID  cards  we  developed  a  simple  programming  device.  The  programmer  is  interfaced  to  a  PC  using  serial  and  PS/2 
(power  source)  ports.  Inside,  there  is  Atmel  89C2051  microcontroller,  which  handles  UART  transmission  and  I2C 
EEPROM  (ID  card)  programming. 

CSU  -  Component 

•     Connection  Manager:  Main  task  to  perform  low-level  blue  tooth  communication 
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•  Data  Analysis  Module:  Performs  the  analysis  of  the  raw  sensor  data 

•  Data  Logger  Module:  Provides  support  for  storing  the  monitored  data. 

•  Visualization  Module:  Provides  user  interface  for  the  supervisors 
CSU  Features 

•  Connection  management 

•  Data  processing 

•  Data  recording 

•  Access  verification 

•  System  maintenance 

Future  Improvements 
DAU 

Small  CMOS  camera  to  monitor  the  operator's  point  of  gaze  single  PCB  (SMD  technology)  low  voltage  ICs 

CSU 

Data  mining  algorithms  advanced  data-  base  encryption  using 
THE  EYE  MOVEMENT  SENSOR 


Figure  9:  The  Eye  Movement  Sensor 

Eye  tracking  is  the  process  of  measuring  either  the  point  of  gaze  (where  one  is  looking)  or  the  motion  of  an  eye 
relative  to  the  head.  An  eye  tracker  is  a  device  for  measuring  eye  positions  and  eye  movement.  Eye  trackers  are  used  in 
research  on  the  visual  system  in  psychology,  in  cognitive  linguistics  and  in  product  design.  There  are  a  number  of  methods 
for  measuring  eye  movement.  The  most  popular  variant  uses  video  images  from  which  the  eye  position  is  extracted. 
Other  methods  use  search  coils  or  are  based  on  the  electrooculogram. 
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PONG 
A  Robot 


Figure  10:  POD 
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Figure  11:  PONG 


LIMITATIONS  AND  FUTURE  ASPECTS 


The  prototype  has  several  limitations,  which  are  not  the  result  of  the  project  deficiency  but  are  rather  caused  by 
the  constraints  imposed  by  the  Project  Kit  and  small  budget. 

The  unique  feature  of  system  relies  on  the  possibility  of  monitoring  the  operator's  higher  brain  functions  involved 
in  the  acquisition  of  the  information  from  the  visual  environment.  The  new  possibilities  can  cover  such  areas  as  industry, 
transportation  (by  air,  by  road  and  by  sea),  military  command  centers  or  operating  theaters  (anesthesiologists). 
It  is  intended  that  the  system  in  its  commercial  release  will  help  avoid  potential  threats  resulting  from  human  errors,  such 
as  weariness,  oversight,  tiredness 

APPLICATIONS 

•  In  retailing  record  and  interpret  customer  movements 

•  In  automobile  industry 
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•  In  video  games 

•  To  create  "Face  Responsive  Display"  and  "Perceptive  Environment"  Generic  control  rooms 

•  Power  station 

•  Flight  control  centres 

•  Operating  theatres 
CONCLUSIONS 

Human  has  tremendous  expectations  from  human  being's  future  and  present.  This  tends  to  research  new  and 
helpful  technologies  which  can  make  the  life  more  comfortable  and  reliable.  This  technology  is  one  of  them  that  can  make 
so.  Artificial  disasters  due  to  consciousness  of  human  brain  can  be  overcome  from  those  accidents. 
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